The Difficulties of Counting Iraq’s War Dead

This publication is now archived.

Introduction

Counting casualties during wartime is a murky and inexact science. In Iraq, in particular, it is made more cumbersome by inadequate census data, poor security, and the lack of an official civilian body count by the U.S. government. A range of epidemiology surveys employing various methodologies have produced staggeringly varied results, the most controversial being one conducted by Johns Hopkins University last fall that found more than 600,000 Iraqis had been killed as a result of violence since the overthrow of Saddam in March 2003. Official estimates by the Iraqi government are closer to the 40,000-50,000 range. The discrepancy in numbers reveals the difficulties of counting casualties during wartime, particularly where civilian centers are the primary arena for battle. Also, given the politically charged atmosphere surrounding the Iraq war in the United States, this opens up any surveys to scrutiny and charges of hidden political biases.

Methodologies Used to Count Casualties

There are three estimates generally used to calculate the number of Iraqi civilians killed by post-war violence:

• The Iraqi government. It relies on data from the Ministry of Health, hospitals, and the Baghdad morgue. The government’s casualty count of violent deaths since April 2003 was roughly fifty thousand as of last June, but the Iraqi health minister estimated last November the total number of civilians dead was somewhere between 100,000 and 150,000. Experts say that number is probably an underestimate because many Muslims in Iraq bury their dead without going through an undertaker or hospital.
• The United Nations. The UN’s figure for the civilian body count, much like the Iraqi government’s, relies on information from hospitals, morgues, and municipalities in Iraq but also takes into account casualty reports from Iraq’s Ministry of Interior; it found that 34,452 Iraqi civilians had died because of violence in 2006.
• IraqBodyCount. This independent organization catalogues its casualties by looking at the number reported killed by confirmed news reports. The group calculates about 62,000 civilians have died from violence in Iraq since April 2003, with the caveat that its findings are low-ball estimates because not all deaths are obviously reported. Some epidemiologists say only around 10 percent of the deaths in Iraq are described in newspapers.
  

How Cluster Sampling Works

Another methodology is random—or cluster—sampling, whereby people are surveyed in select neighborhoods to determine larger patterns of the whole population. That was the method chosen by a team of researchers at the Johns Hopkins University’s Bloomberg School of Public Health (The United Nations uses a similar method to calculate mortality rates).

Their October 2006 study, published in the UK-based journal Lancet, found that a further 2.5 percent—or more than six hundred thousand—of Iraqis had been killed because of post-war violence. The leading cause of death, they find, is gunfire, though casualties from car bombs had also increased. The findings created a storm of controversy because the numbers were astronomically higher than previous casualty estimates. The same group found in October 2004 that Iraq’s death toll had reached one hundred thousand (this survey relied on death figures in a fourteen-month period prior to the March 2003 invasion by way of comparing the data to a similar period after the invasion).

Many statisticians and political scientists defended the Johns Hopkins group’s methodology. The results were peer reviewed and relied not only on random sampling but also death certificates (92 percent of the deaths were certified). Because no reliable data on the number of Iraqis or accurate breakdown of its various ethno-religious groups exists, “unless you do something like an old-time Jerusalem-style census where everybody lines up, you’re not going to get much else,” says Sharyn O’Halloran, a statistician and political science professor at Columbia University.

Criticisms of This Approach

Still, the Lancet study has come under heavy fire from a number of fronts. President Bush told reporters October 11, “I don’t consider it a credible report,” offering up thirty thousand Iraqi deaths as a more accurate estimate. Many critics pan the report’s vagueness. The Lancet figure, Vanity Fair columnist Christopher Hitchens notes in Slate, “does not clearly state, for example, that all these people have actually been killed,” alluding to the “subjective definition of cause of death.” Others criticize the report’s sample size. The researchers randomly chose fifty clusters from sixteen provinces, with each cluster consisting of forty households. The trouble with this methodology, Michael Spagat, a conflict studies expert at the University of London, tells Nature (subscription required) is what he calls “main-street bias”: researchers randomly selected residential streets that crisscross busy thoroughfares in each survey area, which are more prone to car bombs and heavy violence, but left out safer streets off the beaten path.

Also problematic with the survey is determining civilians from combatants. Les Roberts, one of the report’s coauthors, admits “it is probable that many of the household deaths [reported in their study] were indeed combatants.” Another problem is Iraq’s constantly changing demographics, as millions of mostly middle class Iraqis flee for safety in Jordan and Syria. That not only affects the survey’s sample size, says O’Halloran, but also the probability of a civilian getting hit by crossfire, a car bomb, or an aerial assault because there is less density. Respondents also are inherently biased. “There is a tendency to over-inflate the [household casualty] estimate on the part of the respondent because from your position, you want to convey it’s been horrible here,” says O’Halloran. Also, there is the question: What if an entire house was shelled and the whole family perished, as was often the case in insurgency strongholds like Fallujah? These deaths would not be recorded by the Johns Hopkins report unless the survey took into account neighbors or friends of family, which raises the risk of double-counting. Roberts admits their study “is not ideal, but under these settings, it’s the best you can do,” adding that “surveillance is never complete.”

The Politics of Body Counts

Some critics say the Lancet survey’s results were inflated for political reasons, particularly given its publication directly before the November midterm elections. Hitchens accuses Dr. Richard Horton, the journal’s editor, of being a “full-throated speaker at rallies of the Islamist-Leftist alliance that makes up the British Stop the War Coalition.” Roberts dismisses these criticisms.

“No one in public health who studies what causes death is in favor of that thing,” he says. "There’s never been a more controversial study more easy to verify or refute.” That is to say, his researchers found that, on average, one in seven Iraqi households had lost a loved one; the Iraqi government estimates one in eighty have. “You wouldn’t have to call up very many Iraqis to confirm the truth,” says Roberts. Moreover, he argues, if the government’s body counts are true, then South Africa, Colombia, Estonia and a host of other countries would have higher violent death rates than Iraq and American cities like New Orleans or Baltimore would have a higher murder rate than Baghdad.

Part of the problem is that no reliable U.S. government data on Iraqi casualties exists to cross-check against the Lancet study’s findings. The U.S. military, argues Samuel Gardiner, a retired Air Force colonel who has taught at the Naval Defense University, “is in a better position than anyone else [to count casualties] because they have the presence and resources.” But for public relations reasons, he says, they do not collect these statistics. “Their motivation is not to have a body count number out there,” he says. “If there’s a number that appeared on TV every night, it would be that much more difficult to conduct this war. Think of pictures of Iraqis digging mass graves.” The Defense Department does catalogue some instances of collateral damage—for instance, when Iraqis are accidentally targeted and receive claims for financial payouts. But historically, says Gardiner, during wartime the U.S. government, as with most countries, has never formally catalogued civilian casualties (though Roberts says the Center for Disease Control and Prevention, a division of the Department of Health, did conduct some body count estimates after conflicts in Somalia in 1992 and Kosovo in 1999).